Papers with sequence-level tasks
Accelerating BERT Inference for Sequence Labeling via Early-Exit (2021.acl-long)
Copied to clipboard
| Challenge: | Existing early-exit mechanisms are designed for sequence-level tasks, rather than sequence labeling. |
| Approach: | They propose to extend sentence-level early-exit to accelerate inference of PTMs . they propose a token-level mechanism that allows partial tokens to exit early at different layers . |
| Outcome: | The proposed approach can save up to 66%75% inference cost with minimal performance degradation. |
Deriving Entity-Specific Embeddings from Multi-Entity Sequences (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods toward entity-specific prediction involve redundant computation or post-processing outside of the transformer. |
| Approach: | They propose a method for deriving entity-specific embeddings from a multi-entity sequence completely within the transformer, with a loose definition of entity amenable to many problem spaces. |
| Outcome: | The proposed method can be applied to emotion recognition in conversation and player performance projection in baseball and achieve SOTA in both. |